Using Wikipedia to Collect a Corpus for Automatic Definition Extraction: Comparing English and Portuguese Languages

Systems for the detection and extraction of definitions are being developed for different purposes, such as glossaries creation [5, 3], lexical databases [6], ontologies [2], question answering [1], etc. All these systems use annotated corpora to build a set of rules or patterns capable to identify a definition in a different text. The basic structure of a definition should resemble an equation with the definiendum (what is to be defined) on the left hand side and the definiens (the part which is doing the defining) on the right hand side. Between the term defined, and its description there is a a connector, usually a verb or a punctuation symbol. In general, works in this field are restricted in terms of number and types of definitions considered, they are based on specific limited corpora very domain specific, lacking of a general approach. This limitation is due to scarcity of corpora previously annotated with definition information, as these corpora are not usually available and the annotation process constitutes a very expensive task. In this work we propose to use wikipedia as a corpus to extract general domains definitions, that can represent a bootstrap in the construction of a automatic definition extractor. The corpus can be used to draw pattern or extract lexical information characterizing definitions. The convenience of using Wikipedia as font for definition is based on the peculiar structure of its articles, following well-defined rules stated by Wikipedia itself that contributors should follow when write an article. In particular Wikipedia states that the first paragraph of each article should define the topic of the article. In this paper, we focus on the issues arising when extracting a general balanced corpus composed by Wikipedia articles and the size of such a corpus. We presented a study using two different languages, that is Portuguese and English, two different algorithms, and corpora of 5 different sizes.